llm-d Endpoint Picker

Google's GKE Inference Gateway introduces prefix-aware load balancing for LLMs, routing requests to replicas that already hold cached context to avoid reprocessing shared prompt prefixes. This approac…